Search for: All records

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system. 

The COVID-19 pandemic has significantly impacted most countries in the world. Analyzing COVID-19 data from these countries together is a prominent challenge. Under the sponsorship of NSF REU, this paper describes our experience with a ten-week project that aims to guide an REU scholar to develop a physics-guided graph attention network to predict the global COVID- 19 Pandemics. We mainly presented the preparation, implementation, and dissemination of the addressed project. The COVID-19 situation in a country could be dramatically different from that of others, which suggests that COVID-19 pandemic data are generated based on different mechanisms, making COVID-19 data in different countries follow different probability distributions. Learning more than one hundred underlying probability distributions for countries in the world from large scale COVID- 19 data is beyond a single machine learning model. To address this challenge, we proposed two team-learning frameworks for predicting the COVID-19 pandemic trends: peer learning and layered ensemble learning framework. This addressed framework assigns an adaptive physics-guided graph attention network (GAT) to each learning agent. All the learning agents are fabricated in a hierarchical architecture, which enables agents to collaborate with each other in peer-to-peer and cross-layer way. This layered architecture shares the burden of large-scale data processing on machine learning models of all units. Experiments are run to verify the effectiveness of our approaches. The results indicate the proposed ensemble outperforms baseline methods. Besides being documented on GitHub, this work has resulted in two journal papers. 

Human experience involvement in existing operations of airborne Light Detection and Ranging (LIDAR) systems and off-line processing of collected LIDAR data make the acquisition process of airborne LIDAR point cloud less adaptable to environment conditions. This work develops a deep reinforcement learning-enabled framework for adaptive airborne LIDAR point cloud acquisition. Namely, the optimization of the airborne LIDAR operation is modeled as a Markov decision process (MDP). A set of LIDAR point cloud processing methods are proposed to derive the state space, action space, and reward function of the MDP model. A DRL algorithm, Deep Q-Network (DQN), is used to solve the MDP. The DRL model is trained in a flexible virtual environment by using simulator AirSim. Extensive simulation demonstrates the efficiency of the proposed framework. 

The conventional ground penetrating radar (GPR) data analysis methods, which use piecemeal approaches in processing the GPR data formulated in variant formats such as A-Scan, B-Scan, and C-Scan, fail to provide a global view of underground objects on the fly to adapt the operations of GPR systems in the field. To bridge the gap, in this paper, we propose a novel GPR data analysis approach termed “ScanCloud” which is focused on the whole in situ GPR dataset rather than on individual A-Scans, B-Scans or C-Scans. We also study the integration of ScanCloud and a deep reinforcement learning method called deep deterministic policy gradient (DDPG) to adapt the operation of GPR system. The proposed method is evaluated using GPR modeling software called GprMax. Simulation results show the efficacy of ScanCloud and the adaptive GPR system enabled by the integration of ScanCluod and DDPG. 

Context: An Optimizing Performance through Intrinsic Motivation and Attention for Learning theory-based motor learning intervention delivering autonomy support and enhanced expectancies (EE) shows promise for reducing cognitive-motor dual-task costs, or the relative difference in primary task performance when completed with and without a secondary cognitive task, that facilitate adaptive injury-resistant movement response. The current pilot study sought to determine the effectiveness of an autonomy support versus an EE-enhanced virtual reality motor learning intervention to reduce dual-task costs during single-leg balance. Design: Within-subjects 3 × 3 trial. Methods: Twenty-one male and 24 female participants, between the ages of 18 and 30 years, with no history of concussion, vertigo, lower-extremity surgery, or lower-extremity injuries the previous 6 months, were recruited for training sessions on consecutive days. Training consisted of 5 × 8 single-leg squats on each leg, during which all participants mimicked an avatar through virtual reality goggles. The autonomy support group chose an avatar color, and the EE group received positive kinematic biofeedback. Baseline, immediate, and delayed retention testing consisted of single-leg balancing under single- and dual-task conditions. Mixed-model analysis of variances compared dual-task costs for center of pressure velocity and SD between groups on each limb. Results: On the right side, dual-task costs for anterior–posterior center of pressure mean and SD were reduced in the EE group (mean Δ = −51.40, Cohen d  = 0.80 and SD Δ = −66.00%, Cohen d  = 0.88) compared with the control group (mean Δ = −22.09, Cohen d  = 0.33 and SD Δ = −36.10%, Cohen d  = 0.68) from baseline to immediate retention. Conclusions: These findings indicate that EE strategies that can be easily implemented in a clinic or sport setting may be superior to task-irrelevant AS approaches for influencing injury-resistant movement adaptations.